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Detailed Action 

1 . This non- final Office action is responsive to Applicant's preliminary amendments filed 
10 May 2007. Claims 1-47 are presented for examination. The oath submitted 08 June 2007 and 
the specification and drawings submitted 01 September 2006 have been considered and are 
accepted. Claims 1-47 are rejected for the reasons indicated hereinbelow 

Information Disclosure Statement 

2. The information disclosure statements (IDS) submitted on 01 November 2006, 1 1 June 
2007 and 15 January 2009 are in compliance with the provisions of 37 C.F.R. § 1.97. 
Accordingly, the IDS have been considered by the examiner. 

Priority 

3. Acknowledgment is made of Applicant's claim for foreign priority under 35 U.S.C. 
§ 1 19(a)-(d) of Japanese Patent Application No. 2004-059531 filed 03 March 2004. 

Objections - Title & Abstract 

4. The Title submitted 0 1 September 2006 is objected to for not being particularly 
descriptive of the claimed invention. A more descriptive title, such as "Object pose estimation 
and comparison using image sharpness differences," or the like, is suggested. 

5. The abstract submitted 01 September 2006 is also objected to for minor informalities. 
One goal of the abstract is to provide a reader with a concise summary of the key elements of the 
invention such that the reader may quickly determine whether or not the rest of the patent is 
worth considering; i.e. the abstract should be a stand-alone encapsulation of the invention. 
Accordingly, the abstract should not refer back to other portions of the specification by 
incorporating reference numbers to various drawings. Also, the article "an" is repeatedly used 
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where the article "a" would be more appropriate. Please provide a corrected abstract (e.g. "A 
pose estimation and comparison system has a[[n]] pose estimation and comparison unit {20). . ."). 

Claim Rejections - 35 USC §101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

7. Claims 24-29, 32, 35, 38, 41, 44 and 47 are rejected under 35 U.S.C. § 101 as not falling 
within one of the four statutory categories of invention. Independent claims 24, 28, 32, 38 and 
44 each encompass a "program" that causes a computer to implement pose estimation 
processing. However, there is no disclosed relationship between the claimed programs and the 
computer, i.e., the program need not be stored or executed on the computer. A program having 
no disclosed connection to a computer is nonstatutory. Applicant is encouraged to amend each 
of the independent program claims such that a computer, or non-transitory computer-readable 
medium, clearly stores and executes the claimed program, e.g.: " A non-transitory computer- 
readable medium storing a An pose estimation program causing a computer to execute. . ." Each 
of the dependent claims are rejected for their dependence on rejected base claims. 
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Claim Rejections - 35 USC § 112 

8. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

9. Claims 1-47 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Each of similar independent claims 1, 11, 16, 20, 24 and 28 recites the limitation "a 
plurality of comparison images close to the input image" (emphasis added). However, the claim 
phrase " close to the input image" as used across the above-noted independent claims, is a 
subjective phrase of considerable elasticity whose metes and bounds have not been adequately 
defined in the claims, or explicitly defined in the specification. I.e., the artisan of ordinary skill, 
presented with the instant claims, would be unable to determine, with the precision and clarity 
required by §1 12 ^[2, whether "close" as claimed refers to, e.g., spatial proximity, feature 
proximity, etc., or combinations thereof. It is suggested that the claim phrase "close to the input 
image" be clarified using an objectively- verifiable metric that determines closeness as claimed. 
Each of the dependent claims are rejected for their dependence on rejected base claims. 

Each of similar independent claims 30, 31, 32, 36, 37 and 38 recites the limitation 
"pose/illumination." However, the metes and bounds of "pose/illumination" cannot be 
determined from either the claims or the relevant portions of the specification. I.e., the artisan of 
ordinary skill, presented with the instant claims, would be unable to determine, with the 
precision and clarity required by §1 12 ](2, whether "pose/illumination lose" as claimed refers to, 
e.g., "pose and illumination," or "pose or illumination" etc. It is suggested that the claim phrase 
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"pose/illumination" be clarified by inserting either "and" or "or" between pose and illumination. 
Each of the dependent claims are rejected for their dependence on rejected base claims. 

Each of similar independent claims 33, 39, and 45 recites the limitation "the two- 
dimensional image having the highest similarity." However, there is insufficient antecedent 
basis for referring back to "the two-dimensional image having the highest similarity" since no 
such image has been established in the claims. It is suggested that independent claims 33, 39 and 
45 be amended to recite "a two-dimensional image having the highest similarity" Each of the 
dependent claims are rejected for their dependence on rejected base claims. 

Claim 41 recites the limitation of "if the two-dimensional image having the highest 
similarity in brightness is not clear . . ." However, the metes and bounds of "clear" cannot be 
determined from either the claim or the relevant portions of the specification. I.e., the artisan of 
ordinary skill, presented with the instant claims, would be unable to determine, with the 
precision and clarity required by § 1 12 ^f2, what metrics are being used to define clarity as 
claimed. It is suggested that "clear" be replaced with "sharp," as found in similar claims 39 and 
40. 

Any amendment to the claims must be commensurate in scope with the corresponding 
disclosure. 
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Claim Rejections - 35 USC §103 

10. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 

11. Claims 1-3, 6-8, 1 1-13, 16-47 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ishiyama U.S. Pre-Grant Application Publication No. 2003/0035098 ("Ishiyama") in view 
of Kohmo U.S. Pre-Grant Application Publication No. 2003/0161535 ("Kohmo"). 

Regarding similar independent claims 1 (system), 1 1 (system), 1 6 (method), 20 (method), 
24 (program) and 28 (program) Ishiyama teaches a pose estimation system for performing object 
pose estimation by comparing an input image with a three-dimensional object model (see Figs. 
3-5 and ^0010 and 0026 for an overview of Ishiyama 's pose estimation system), the pose 
estimation system comprising: 

i. an pose candidate decision unit for generating at least one pose candidate (see Ishiyama 
Fig. 3 as described in \\0040-41 where a pose estimation routine is disclosed); 

ii. a comparison image generation unit for generating, according to the generated pose 
candidate, a plurality of comparison images close to the input image, while projecting the 
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three-dimensional object model to a two-dimensional image (see Ishiyama Fig. 3 as 
described in ^0044-45 where images are projected onto a pose candidate); 

iii. a difference calculator for calculating a plurality of differences (see Ishiyama Fig. 3 as 
described in ^0045-46 where differences between the projected images and the pose 
candidate are calculated); and 

iv. a determination unit for selecting a comparison image having the smallest difference 
among the plurality of differences and estimating an optimal pose based on the selected 
comparison image {see Ishiyama Fig. 3 item 310 as described in ^0047 where the image 
candidate whose distance to the input image is smallest is selected). 

Ishiyama is silent on the system further comprising a first sharpness extraction unit for extracting 
a first sharpness amount reflecting the sharpness from each of the plurality of comparison 
images, and a weighted difference calculator, as claimed. However, Kohmo teaches a similar 
image similarity determination system comprising using a measure of sharpness (e.g., a degree of 
camera blur when acquiring an image) in similarity determination processing (see Kohmo 
\\0065-66, where weighted sharpness (i.e. blurring) values are used to determine image 
similarity). 

The ordinarily-skilled artisan, starting with the pose estimation system of Ishiyama, 
would have appreciated the benefits of using a weighted measure of sharpness to determine pose 
similarity, as proposed by Kohmo in ]ffl0065-66. The resulting combination of Ishiyama and 
Kohmo would achieve the predictable and expected benefits of increasing the accuracy of the 
pose estimation by incorporating a measure of image sharpness. The artisan of ordinary skill 
would have been motivated to combine Kohmo with Ishiyama, as proposed above, at least 
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because both Kohmo and Ishiyama are drawn to similar image matching systems, and so the 
similarities between Kohmo and Ishiyama invite further modifications between the two. 

Therefore, a person having ordinary skill in the image processing arts at the time of the 
invention would have found it obvious to combine the weighted sharpness values of Kohmo, 
with the pose estimation system of Ishiyama to achieve the well-known and expected benefits of 
increasing the accuracy of the pose estimation by incorporating a measure of image sharpness. 

Regarding similar claims 2 (system), 12 (system), 17 (method), 21 (method), 25 
(program) and 29 (program) Ishiyama and Kohmo teach the method as indicted re claim 1 above, 
but are silent on the method further comprising a second sharpness extraction unit for extracting 
a second sharpness amount reflecting the sharpness from the input image, wherein the weighted 
difference calculator calculates a plurality of weighted differences by weighting the difference 
between the second sharpness amount of the input image and the first sharpness amount of each 
of the comparison images to the difference between the input image and the comparison image. 
However, official notice is taken to note that the uses and benefits of using a second sharpness 
extractor are well known and expected in the image processing arts (and Kohmo already teaches 
a first sharpness extractor, as indicated re claim 1 above). It would have been obvious to the 
artisan of ordinary skill at the time of the invention to use the results from a second sharpness 
extractor within the system of Ishiyama and Kohmo as described re claim 1 above, to achieve the 
known and expected uses and benefits of increasing the accuracy of the resulting pose 
estimation 
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Regarding similar claims 3 (system), 8 (system), 13 (system), Ishiyama and Kohmo teach 
the method as indicted re claim 1 above, but are silent on the method further comprising wherein 
the weight becomes higher as the sharpness of the image becomes higher in the weighted 
difference calculation. However, official notice is taken to note that the uses and benefits of 
using a variable weight are well known and expected in the image processing arts (and Kohmo 
already teaches a weighted first sharpness value, as indicated re claim 1 above). It would have 
been obvious to the artisan of ordinary skill at the time of the invention to use a variable 
weighted sharpness result within the system of Ishiyama and Kohmo as described re claim 1 
above, to achieve the known and expected uses and benefits of increasing the accuracy of the 
resulting pose estimation. 

Regarding similar claims 6 (system), 7 (system), 26 (program) and 27 (program), 
Ishiyama and Kohmo teach the method as indicted re claim 1 above, but are silent on the method 
further comprising wherein the determination unit further performs object comparison by 
comparing the minimum weighted difference of the estimated optimal pose with a predetermined 
threshold value. However, official notice is taken to note that the uses and benefits of using a 
predetermined threshold in pose estimation are well known and expected in the image processing 
arts. It would have been obvious to the artisan of ordinary skill at the time of the invention to 
use a predetermined threshold within the system of Ishiyama and Kohmo as described re claim 1 
above, to achieve the known and expected uses and benefits of increasing the accuracy of the 
resulting pose estimation 

Regarding similar claims 18 (method) and 19 (method), Ishiyama further teaches 
selecting a comparison image having the smallest weighted difference among the plurality of 
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weighted differences; and estimating an optimal pose based on the selected comparison image 
(see Ishiyama Fig. 3 item 310 as described in \0047 where the image candidate whose distance 
to the input image is smallest is selected). 

Regarding similar claims 22 (method) and 23 (method), Ishiyama further teaches 
difference calculation method according to claim 20, further comprising: performing comparison 
by comparing the plurality of weighted differences obtained by the calculation (see Ishiyama 
Fig. 3 item 310 as described in \0047 where the image candidate whose distance to the input 
image is smallest is selected). 

Regarding similar independent claims 30 (method), 36 (method) and 42 (method) 
Ishiyama teaches object pose/illumination estimation method (see Figs. 3-5 and \\0010 and 
0026 for an overview of Ishiyama 's pose estimation system) for estimating at least one of the 
pose and the illumination conditions of an object (see Ishiyama Fig. 3 as described in ^0040-41 
where a pose estimation routine is disclosed) by generating a two-dimensional image of the 
object while changing at least one of the pose and the illumination conditions of the object with 
the use of a three-dimensional model of the objectfsee Ishiyama Fig. 3 as described in ^0044- 
45 where images are projected onto a pose candidate). 

Ishiyama is silent on the system wherein sharpness of the generated two-dimensional 
image is reflected in the similarity, as claimed. However, Kohmo teaches a similar image 
similarity determination system comprising using a measure of sharpness (e.g., a degree of 
camera blur when acquiring an image) in similarity determination processing (see Kohmo 
y^0065-66, where weighted sharpness (i.e. blurring) values are used to determine image 
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similarity). Motivation to combine Ishiyama and Kohmo to achieve the results of claim 30, 36 
and 42 can be found re similar claim 1 above. 

Regarding similar independent claims 31 (system), 37 (system) and 43 (system) Ishiyama 
teaches an image generation unit for generating a two-dimensional image of an object (see Figs. 
3-5 and ^0010 and 0026 for an overview of Ishiyama 's pose estimation system) while changing 
at least one of the pose and the illumination conditions of the object with the use of a three- 
dimensional model of the object (see Ishiyama Fig. 3 as described in ^0040-41 where a pose 
estimation routine is disclosed); a calculator for calculating the similarity by comparing the 
generated two-dimensional image with an input image (see Ishiyama Fig. 3 item 310 as 
described in \0047 where the image candidate whose distance to the input image is smallest is 
selected). 

Ishiyama is silent on the system further comprising an extraction unit for extracting 
sharpness from the generated two-dimensional image and reflecting the extracted sharpness in 
the calculation; and a determination unit for estimating at least one of the pose and the 
illumination conditions based on the calculation result of the calculator, as claimed. However, 
Kohmo teaches a similar image similarity determination system comprising using a measure of 
sharpness (e.g., a degree of camera blur when acquiring an image) in similarity determination 
processing (see Kohmo ^0065-66, where weighted sharpness (i.e. blurring) values are used to 
determine image similarity). Motivation to combine Ishiyama and Kohmo to achieve the results 
of claim 31, 37 and 43 can be found re similar claim 1 above. 

Regarding similar independent claims 32 (system), 38 (system) and 44 (system) Ishiyama 
teaches image generation processing for generating a two-dimensional image of the object (see 
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Figs. 3-5 and ^0010 and 0026 for an overview oflshiyama 's pose estimation system) while 
changing at least one of the pose and the illumination conditions of the object with the use of a 
three-dimensional model of the object (see Ishiyama Fig. 3 as described in ^0040-41 where a 
pose estimation routine is disclosed); calculation processing for calculating the similarity by 
comparing the generated two-dimensional image with an input image (.see Ishiyama Fig. 3 item 
310 as described in \0047 where the image candidate whose distance to the input image is 
smallest is selected). 

Ishiyama is silent on the system further comprising extraction processing for extracting 
sharpness from the generated two-dimensional image, and reflecting the extracted sharpness in 
the calculation, as claimed. However, Kohmo teaches a similar image similarity determination 
system comprising using a measure of sharpness (e.g., a degree of camera blur when acquiring 
an image) in similarity determination processing (see Kohmo ^0065-66, where weighted 
sharpness (i.e. blurring) values are used to determine image similarity). Motivation to combine 
Ishiyama and Kohmo to achieve the results of claim 32, 38 and 44 can be found re similar claim 
1 above. 

Regarding similar claims 33 (method), 34 (system), 35 (program), 39 (method), 40 
(system), 41 (program), 45 (method), 46 (system) and 47 (program), Ishiyama and Kohmo teach 
a pose estimation system but are silent on the system further comprising wherein, if the two- 
dimensional image having the highest similarity in brightness is not, sharp the estimation to the 
input image is not employed. However, official notice is taken to note that the uses and benefits 
of varying the sharpness estimation based upon similarity are well known and expected in the 
image processing arts. It would have been obvious to the artisan of ordinary skill at the time of 
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the invention to vary the sharpness estimation based upon similarity within the system of 
Ishiyama and Kohmo as described above, to achieve the known and expected uses and benefits 
of increasing the accuracy of the resulting pose estimation. 



Claim Objections 

12. Claims 4-5, 9-10, and 14-15 would be allowable if rewritten to overcome the rejection 
under 35 U.S.C. §101 and 35 U.S.C. §1 12 ^2 set forth in this Office action. 

13. The following is a statement of reasons for the indication of allowable subject matter: 
Regarding similar dependent claims 4-5, 9-10, and 14-15 the prior art of record, alone or in 
combination, fails to fairly teach of suggest the limitations of: wherein the first and second 
sharpness amounts are defined by a ratio of a number of pixels whose edge intensity is a 
threshold value or higher to the total number of pixels, a range of brightness values, dispersion of 
brightness values, or a number of characteristic points, and wherein the first and second 
sharpness amounts are defined by an edge image or a characteristic point, as recited in the claims 
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Comment on 35 USC §101 

14. Independent claims 16, 28, 30, 36 and 42 are "process" claims and have been analyzed in 
light of Bilski et al v. Kappos and the relevant guidance [2] , [3] . The independent claims are 
not directed to an abstract idea at least because the independent claims encompasses more than 
just a statement of concept, and describe a particular solution to the pose estimation problem to 
be solved. Furthermore, the independent claims tangibly implement the methods, necessarily 
using image processing equipment. Additionally, the performance of the pose estimation steps is 
observable (e.g. by observing the similarity between the pose candidate and the comparison 
images) and verifiable (e.g. by verifying the resulting similarity between the pose candidate and 
the comparison images). 

Also, independent claims 16, 28, 30, 36 and 42 are found to pass at least the machine 
prong of the machine or transformation test, at least because the method of claims 16, 28, 30, 36 
and 42 necessarily requires the use of a particular machine (e.g. a processor) to implement the 
steps of the method. 

Therefore, based upon consideration of all the relevant factors [3] with respect to the claim 
as a whole, independent claims 16, 28, 30, 36 and 42, and all claims dependent therefrom, are 
not directed to an abstract idea. 



[1] See Bilski et al v. Kappos (SCt. 08-964), 

[2] See Memorandum to the Examining Corps, Regarding the Supreme Court Decision in Bilski v. Kappos, issued 
June 28, 2010, a\ i < u > t /vo gov/patenis/law/exam/memoranda. jsp 

[3] See Interim Guidance for Determining Subject Matter Eligibility for Process Claims in View of Bilski v. Kappos, 
Federal Registrar, Vol. 75, No. 143, issued July 27, 2010 
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Contact 

1 5 . Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to Steve Koziol: 

• phone 571.270.1844, fax 571.270.2844. or e-mail stepheti.koziol@uspto.eov . 

Typically, the examiner can be reached Monday - Friday 9:00 - 5:30 ET. For e-mail 

communications, please note MPEP 502.03, which states, in relevant part, "[w]ithout a written 

authorization by applicant in place, the USPTO will not respond via Internet e-mail to any 

Internet correspondence which contains information subject to the confidentiality requirement as 

set forth in 35 U.S.C. § 122." A sample authorization form which may be used by applicant can 

be found in MPEP 502.03 section II. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 

supervisor, Bhavesh Mehta can be reached at (571) 272-74 53. Customer Service can be reached 

at (571) 272-2600. The fax number for the organization where this application or proceeding is 

assigned is (571) 273-8300. Information regarding the status of an application may be obtained 

from the Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. Status 

information for unpublished applications is available through Private PAIR only. For more 

information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions 

on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 

9197 (toll-free). 

/Stephen R Koziol/ 
Examiner, Art Unit 2624 
10 August 2010 



